Skip to content

Add AI skills to drive end-to-end tests#25354

Open
crazytonyli wants to merge 1 commit intotrunkfrom
explore-e2e-tests
Open

Add AI skills to drive end-to-end tests#25354
crazytonyli wants to merge 1 commit intotrunkfrom
explore-e2e-tests

Conversation

@crazytonyli
Copy link
Contributor

Description

This PR adds two skills:

  1. ios-sim-navigation. This skill is basically a reference of WDA, which Claude Code can use to interact with iOS simulators.
  2. ai-test-runner. This skill describes a workflow for running end-to-end tests on iOS simulators using the ios-sim-navigation skill.

You can use ios-sim-navigation to verify your code changes. For example, you can add this to your prompt:

After completing the code changes, use the general-purpose agent with the ios-sim-navigation skill to verify the changes on the iPhone 17 Simulator.

You can try the ai-test-runner locally using a prompt like this:

Run the test case at Tests/AgentTests/ui-tests/create-blank-page.md on the iPhone 17 simulator. Here are the test site credentials: [url], [username], [application-password].

I have translated some UI tests into test cases in Tests/AgentTests/ui-tests. If you want, you can ask Claude Code to run the full test suite, which takes 40+ minutes. You can also ask Claude Code to run on two simulators simultaneously, which halves the runtime.

@crazytonyli crazytonyli requested review from jkmassel and kean March 9, 2026 08:17
@dangermattic
Copy link
Collaborator

dangermattic commented Mar 9, 2026

1 Warning
⚠️ This PR is larger than 500 lines of changes. Please consider splitting it into smaller PRs for easier and faster reviews.

Generated by 🚫 Danger

@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 9, 2026

@crazytonyli crazytonyli added the Tooling Build, Release, and Validation Tools label Mar 9, 2026
@crazytonyli crazytonyli added this to the 26.8 milestone Mar 9, 2026
@wpmobilebot
Copy link
Contributor

App Icon📲 You can test the changes from this Pull Request in Jetpack by scanning the QR code below to install the corresponding build.
App NameJetpack
ConfigurationRelease-Alpha
Build Number31375
VersionPR #25354
Bundle IDcom.jetpack.alpha
Commit4ce4ab4
Installation URL6pjn4fq5kcv9g
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot
Copy link
Contributor

App Icon📲 You can test the changes from this Pull Request in WordPress by scanning the QR code below to install the corresponding build.
App NameWordPress
ConfigurationRelease-Alpha
Build Number31375
VersionPR #25354
Bundle IDorg.wordpress.alpha
Commit4ce4ab4
Installation URL68ui02in8rn88
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot
Copy link
Contributor

🤖 Build Failure Analysis

This build has failures. Claude has analyzed them - check the build annotations for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Tooling Build, Release, and Validation Tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants